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SUBJECT :  Report  on  Acquisition  of  the  Advanced  Deployable  System 
(Report  No.  D-2003'004) 


We  are  providing  this  report  for  information  and  use.  We  considered 
management  comments  on  a  draft  of  diis  report  when  preparing  the  final  report. 

Commraits  on  the  draft  rqport  conformed  to  the  requirements  of  DoD 
Directive  7650.3  and  left  no  unresolved  issues.  Therefore,  no  additional  comments  are 
required. 
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Report  No,  D-2003-004  October  3,  2002 

(Project  No.  D2002AE-0003) 

Acquisition  of  the  Advanced  Deployable  System 


Executive  Summary 


Who  Should  Read  This  Report  and  Why?  This  report  should  be  read  by  all  who  are 
interested  in  the  aequisition  of  the  Navy’s  Advaneed  Deployable  System  (the  System). 
The  report  addresses  aequisition  issues  that  require  higher  management  attention  before 
the  System  program  should  be  allowed  to  progress  further  through  the  aequisition 
proeess. 

Background.  The  System,  a  Navy  Aequisition  Category  II  program,  is  a  next- 
generation,  ship-deployable,  undersea  surveillanee  system  that  is  designed  to  operate  in 
littoral  waters.  The  System  is  linked  to  a  land  faeilily  for  data  proeessing,  evaluation,  and 
reporting.  The  System  will  be  used  to  eonduct  missions,  such  as  threat  port  surveillance, 
friendly  port  protection,  area  defense,  area  sanitization,  and  strategic  indications  and 
warnings.  The  System  will  have  the  ability  to  be  installed  overtly  or  covertly,  depending 
on  the  needs  of  the  Joint  Task  Force  Commander.  The  program  office’s  estimate 
includes  $793.7  million  for  research,  development,  test,  and  evaluation  for  all  four  blocks 
of  the  evolutionary  acquisition  strategy  and  $785  million  for  procurement  for  the  first  two 
blocks. 

Results.  Overall,  the  System  program  warrants  attention  in  the  areas  of  acquisition 
category  designation,  earned  value  management,  and  documentation  before  it  enters  the 
full-rate  production  phase  of  the  acquisition  process. 

•  The  System  program  manager  did  not  inform  the  Assistant  Secretary  of  the 
Navy  (Research,  Development,  and  Acquisition)  that  the  oversight  of  the 
program  should  be  raised  to  the  level  of  an  acquisition  category  I  program. 

As  a  result,  acquisition  management  oversight  was  not  provided 
commensurate  with  that  required  for  an  acquisition  category  I  program. 
Designating  the  System  as  an  acquisition  category  I  program  should  provide 
the  oversight  necessary  for  an  acquisition  program  of  this  magnitude 
(finding  A). 

•  The  program  office  did  not  apply  standard  estimate-at-completion  formulas  in 
calculating  earned  value  management  computations  and  did  not  request  the 
Defense  Contract  Management  Agency  to  review  earned  value  management 
calculations.  As  a  result,  the  program  office’s  estimate  at  completion  was 
significantly  lower  than  the  estimate  at  completion  calculated  using  standard 
formulas  and  did  not  provide  decision  makers  with  accurate  data  on  estimated 
contract  cost  overruns.  Revising  the  current  draft  memorandum  of  agreement 
between  the  program  office  and  the  Defense  Contract  Management  Agency  to 
include  Defense  Contract  Management  Agency  oversight  of  contractor  earned 
value  management  calculations  will  help  provide  accurate  earned  value 
management  data  to  decision  makers  (finding  B). 


•  The  System  program  offiee  had  not  completed  actions  to  update  the 

acquisition  strategy;  cost  analysis  requirements  description;  life-cycle  cost 
estimate;  command,  control,  communications,  computers  and  intelligence 
support  plan;  and  programmatic  environmental,  safety,  and  health  evaluation 
plan  as  required  to  show  the  current  status  of  the  program.  As  a  result,  the 
program  manager  did  not  have  up-to-date  acquisition  documentation  needed 
to  effectively  manage  program  cost  and  performance  and  acquisition  decision 
makers  could  not  make  fully  informed  investment  decisions  (finding  C). 

For  details  of  the  audit  results,  see  the  Findings  section  of  the  report. 

Management  Comments.  The  Navy  concurred  with  the  audit  findings  and  all 
recommendations;  therefore,  no  further  comments  are  required.  The  Findings  section  of 
the  report  contains  a  summary  of  the  management  comments  and  the  Management 
Comments  section  contains  the  complete  text  of  management  comments. 
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Advanced  Deployable  System 


Background 


The  Advanced  Deployable  System  (ADS),  a  Navy  Acquisition  Category  II 
program,  is  a  next-generation,  ship-deployable,  undersea  surveillance  system  that 
is  designed  to  operate  in  littoral  waters.  The  ADS  is  linked  to  a  land  facility  for 
data  processing,  evaluation,  and  reporting.  The  system  consists  of  three  major 
segments: 

•  The  underwater  segment,  which  provides  the  acoustic  sensing,  cabling, 
and  telemetry. 

•  The  processing  and  analysis  segment,  which  performs  the  signal 
processing;  display  functions;  and  command,  control,  communications, 
and  intelligence  (C^^I). 

•  The  mission  support  segment,  which  provides  system  support  and 
installation  functions. 

The  ADS  will  provide  the  Joint  Task  Force  Commander  with  the  capability  to 
rapidly  and  flexibly  install,  from  a  variety  of  possible  platforms,  an  acoustic, 
littoral  surveillance  system  that  provides  a  complete  undersea  picture  and  target 
locations  to  tactical  systems.  The  ADS  will  be  used  to  conduct  missions,  such  as 
threat  port  surveillance,  friendly  port  protection,  area  defense,  area  sanitization, 
and  strategic  indications  and  warnings.  The  system  will  have  the  ability  to  be 
installed  overtly  or  covertly,  depending  on  the  needs  of  the  Joint  Task  Force 
Commander.  The  ADS  program  includes  two  platforms  for  deployment.  Platform 
Alpha  and  Platform  Bravo.  Appendix  B  provides  definitions  of  technical  terms 
used  in  this  report. 

At  the  engineering  and  manufacturing  decision  meeting  held  on  February  10, 
2000,  the  Assistant  Secretary  of  the  Navy  (Research,  Development,  and 
Acquisition),  the  milestone  decision  authority,  approved  the  Platform  Alpha 
portion  of  the  ADS  program  to  enter  the  engineering  and  manufacturing 
development  phase  of  the  acquisition  process.  The  Platform  Bravo  portion  was  to 
remain  in  the  program  definition  and  risk  reduction  phase,  pending 
accomplishment  of  an  at-sea  demonstration  and  a  program  review  in  2001 .  The 
April  2001  review  addressed  a  major  program  restructuring,  revision  of  the 
acquisition  decision  memorandum,  and  a  requirement  for  a  followup  review  in  the 
fourth  quarter  of  FY  2002.  The  major  program  restructuring  resulted  in  the 
program  office  using  a  four-block  evolutionary  acquisition  strategy  to  develop  the 
ADS.  The  program  office’s  estimate  includes  $793.4  million  for  research, 
development,  test,  and  evaluation  for  all  four  blocks  of  the  evolutionary 
acquisition  strategy  and  $785  million  for  procurement  for  the  first  two  blocks. 

Major  Program  Revisions 


The  ADS  program  has  gone  through  a  number  of  changes  over  its  life. 
Originally,  the  ADS  program  office  was  to  pursue  Platform  Alpha  and  Platform 
Bravo  concurrently  to  provide  the  Navy  with  multiple  deployment  options.  In 
February  2000,  because  the  milestone  decision  authority  approved  only  the 
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Platform  Alpha  for  engineering  and  manufaeturing  development,  the  program 
structure  concentrated  on  that  platform.  Subsequently,  a  program  budget 
reduction  made  it  apparent  that  engineering  and  manufacturing  development  for 
both  platforms  could  not  be  pursued  concurrently,  regardless  of  the  results  of  the 
Platform  Bravo  program  review  in  2001.  In  April  2001,  the  Office  of  the  Chief  of 
Naval  Operations  directed  a  restructure  of  the  program  to  concentrate  efforts  on 
one  version  of  deployment  from  Platform  Bravo,  followed  by  two  additional 
versions  of  deployment  from  Platform  Bravo  and  one  version  of  deployment  from 
Platform  Alpha.  As  a  result,  the  program  office  devised  a  four-block  evolutionary 
acquisition  strategy.  The  Assistant  Secretary  of  the  Navy  (Research, 
Development,  and  Acquisition)  approved  the  restructuring  in  May  2001,  and 
required  that  a  followup  program  review  be  held  in  approximately  1  year, 
allowing  time  for  the  modification  and  approval  of  the  operational  requirements 
document  (ORD)  and  development  of  a  new  acquisition  program  baseline 
agreement.  The  followup  program  review  is  scheduled  for  the  fourth  quarter  of 
FY  2002  because  of  delays  in  the  ORD  approval  process. 

Objectives 


The  audit  objective  was  to  evaluate  the  overall  management  of  ADS.  Because  the 
program  was  in  the  engineering  and  manufacturing  development  phase,  we 
determined  whether  management  was  cost-effectively  developing  and  readying 
the  system  for  the  full-rate  production  phase  of  the  acquisition  process.  In 
addition,  we  evaluated  the  management  control  program  as  it  related  to  the  audit 
objectives.  See  Appendix  A  for  a  discussion  of  the  audit  scope,  methodology,  the 
review  of  the  management  control  program,  and  prior  coverage  related  to  the 
audit  objectives. 
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A.  Acquisition  Category 

The  ADS  program  manager  did  not  inform  the  Assistant  Seeretary  of  the 
Navy  (Researeh,  Development,  and  Aequisition)  that  the  oversight  of  the 
program  should  be  raised  to  the  level  of  an  aequisition  eategory  I  program. 
This  eondition  oeeurred  beeause  the  program  manager  believed  that  eost 
information  showing  that  the  program  had  exeeeded  established  thresholds 
for  researeh,  development,  test,  and  evaluation  had  been  provided  to  the 
Assistant  Seeretary  of  the  Navy  and  that  no  further  aetion  on  his  part  was 
required.  As  a  result,  aequisition  management  oversight  was  not  provided 
eommensurate  with  that  required  for  an  aequisition  eategory  I  program. 

Acquisition  Category  Guidance 


DoD  defines  an  aequisition  eategory  as  an  attribute  of  an  aequisition  program  that 
determines  the  program’s  level  of  review,  deeision  authority,  and  applieable 
proeedures.  Acquisition  category  I  programs  include  two  subcategories: 
acquisition  category  ID  programs  where  the  milestone  decision  authority  is  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics,  and 
acquisition  category  IC  programs  where  the  milestone  decision  authority  is  the 
Component  Acquisition  Executive. 

DoD  Instruction  5000.2,  “Operation  of  the  Defense  Acquisition  System,”  April  5, 
2002, ‘  requires  programs  for  which  estimated  expenditures  for  research, 
development,  test,  and  evaluation  are  more  than  $365  million  in  FY  2000  constant 
dollars  or  for  which  procurement  expenditures  are  more  than  $2.19  billion  in 
FY  2000  constant  dollars  be  classified  as  acquisition  category  I  major  programs. 
The  Instruction  requires  the  DoD  Component  to  notify  the  Under  Secretary  of 
Defense  for  Acquisition,  Technology,  and  Logistics  when  cost  growth  or  a  change 
in  acquisition  strategy  results  in  reclassifying  a  formerly  lower  acquisition 
category  program  as  an  acquisition  category  I  program.  Changes  in  the 
acquisition  category  level  should  be  reported  as  soon  as  the  DoD  Component 
suspects,  within  reasonable  confidence,  that  the  program  is  within  a  10-percent 
encroachment  of  the  next  acquisition  category  level.  The  Under  Secretary  of 
Defense  for  Acquisition,  Technology,  and  Logistics  designates  an  acquisition 
category  reclassification. 

ADS  Research,  Development,  Test,  and  Evaluation 


The  ADS  program  manager  did  not  recommend  to  the  Assistant  Secretary  of  the 
Navy  (Research,  Development,  and  Acquisition)  that  oversight  of  the  program  be 
raised  to  the  level  of  an  acquisition  category  I  program.  Since  the  Navy 
established  the  ADS,  the  estimated  program  costs  have  grown.  The  ADS  program 
exceeded  the  research,  development,  test,  and  evaluation  thresholds  for  an 
acquisition  category  II  program,  after  the  Assistant  Secretary  of  the  Navy 


'Prior  versions  of  DoD  Instruction  5000.2  also  contain  the  requirements  outlined  in  this  report. 
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(Research,  Development,  and  Acquisition)  approved  the  program  to  enter  the 
engineering  and  manufacturing  development  phase  in  February  2000.  At  the 
engineering  and  manufacturing  development  milestone  decision  review,  the 
program  office  estimated  that  the  program  would  require  $134.6  million  for 
research,  development,  test,  and  evaluation.  The  estimate  was  based  on  Platform 
Alpha  and  excluded  costs  for  Platform  Bravo.  The  subsequent  decision  to 
restructure  the  program  and  use  a  four-block  acquisition  strategy  significantly 
increased  the  program’s  estimated  cost  for  research,  development,  test,  and 
evaluation.  The  Assistant  Secretary  of  the  Navy  (Research,  Development,  and 
Acquisition)  approved  the  program  restructure  in  the  updated  engineering  and 
manufacturing  development  acquisition  decision  memorandum  dated  May  25, 
2001 .  As  of  June  2002,  the  program  office  estimated  that  research,  development, 
test,  and  evaluation  costs  for  the  four-block  acquisition  strategy  will  total 
$793.4  million. 

The  ADS  program  manager  was  aware  that  the  ADS  program  met  the  criteria  for 
an  acquisition  category  I  program,  and  stated  that  research,  development,  test,  and 
evaluation  estimates  were  provided  to  the  Assistant  Secretary  of  the  Navy 
(Research,  Development,  and  Acquisition)  as  part  of  the  normal  reporting 
process.  The  program  manager  felt  that  the  cost  information  showing  that  the 
program  had  exceeded  the  acquisition  category  II  research,  development,  test,  and 
evaluation  threshold  of  $365  million,  as  established  in  DoD  Instruction  5000.2, 
was  sufficient  notification  to  the  Assistant  Secretary  of  the  Navy  and  that  no 
further  action  or  recommendation  on  his  part  was  required. 

Personnel  in  the  Office  of  the  Assistant  Secretary  of  the  Navy  confirmed  that  the 
program  manager  had  provided  ADS  research,  development,  test,  and  evaluation 
cost  figures.  However,  because  the  program  manager  did  not  specifically 
reference  the  acquisition  category  II  cost  threshold,  the  personnel  were  not  alerted 
that  the  cost  threshold  for  an  acquisition  category  II  program  had  been  exceeded. 
After  being  informed  of  the  cost  threshold  breach.  Navy  personnel  agreed  that  the 
ADS  program  should  be  reclassified  as  an  acquisition  category  I  program. 
Additionally,  the  personnel  stated  that  they  applied  lessons  learned  from  the  ADS 
program  to  other  Navy  programs  and  took  proactive  measures  to  identify  other 
programs  that  may  have  exceeded  their  acquisition  category  thresholds.  As  a 
result,  another  Navy  program  was  identified  and  corrective  action  was  initiated. 

Acquisition  Category  I  Oversight 


Acquisition  management  oversight  of  ADS  was  not  provided  commensurate  with 
that  required  for  an  acquisition  category  I  program.  By  not  recommending  that 
the  ADS  program  be  classified  as  an  acquisition  category  I  program,  the  program 
office  avoided  additional  requirements  designed  to  provide  decision  makers  and 
Congress  with  greater  visibility  into  program  cost,  schedule,  and  performance  for 
programs  of  this  magnitude.  DoD  Regulation  5000.2-R  requires  that  program 
managers  for  acquisition  category  I  programs  to  prepare  selected  acquisition 
reports,  unit  cost  reports,  contractor  cost  data  reports,  manpower  estimates,  and  to 
obtain  an  independent  life-cycle  cost  estimate.  The  ADS  program  will  benefit 
from  the  additional  oversight  resulting  from  those  reporting  requirements. 
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Recommendation  and  Management  Comments 


A,  We  recommend  that  the  Assistant  Secretary  of  the  Navy  (Research, 
Development,  and  Acquisition)  request  that  the  Under  Secretary  of  Defense 
(Acquisition,  Technology,  and  Logistics)  designate  the  Advanced  Deployable 
System  as  an  acquisition  category  I  program. 

Management  Comments,  The  Deputy  Assistant  Secretary  of  the  Navy 
(Planning,  Programming  and  Resources)  concurred,  pending  a  program  review 
requested  by  the  Assistant  Secretary  of  the  Navy  (Research,  Development,  and 
Acquisition).  The  review  will  be  completed  by  October  3 1 ,  2002.  By  that  point, 
the  Assistant  Secretary  will  determine  the  appropriate  acquisition  category  for  the 
Advanced  Deployable  System  and,  if  appropriate,  request  that  the  Under 
Secretary  of  Defense  (Acquisition,  Technology,  and  Logistics)  designate  the 
system  as  an  acquisition  category  I  program. 
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B.  Earned  Value  Management 

The  program  office  did  not  apply  standard  estimate-at-completion 
formulas  in  calculating  earned  value  management  computations  because  it 
did  not  believe  that  the  standard  formulas  accurately  reflected  the  unique 
aspects  of  the  ADS  program.  Also,  the  program  office  did  not  request  the 
Defense  Contract  Management  Agency  to  review  earned  value 
management  calculations  because  it  believed  its  personnel  were  better 
equipped  to  provide  oversight  of  the  research  and  development.  As  a 
result,  the  program  office’s  EAC  was  significantly  lower  than  the  EAC 
calculated  using  standard  formulas  and  did  not  provide  decision  makers 
with  accurate  data  on  estimated  contract  cost  overruns. 

Earned  Value  Management  Guidance 


The  “Earned  Value  Management  Implementation  Guide,”  Revision  1,  October  3, 
1997,  provides  guidance  to  be  used  during  the  implementation  and  surveillance  of 
contractor  earned  value  management  systems  (EVMS).  The  Guide  states  that 
earned  value  management  is  a  tool  that  allows  program  managers  to  manage 
technical,  cost,  and  schedule  progress  on  their  contracts.  Implementation  of  an 
EVMS  integrates  the  cost,  schedule,  and  technical  aspects  of  the  contract  and 
provides  the  program  manager  with  contractor  cost  and  schedule  performance 
data  that: 


•  relate  time -phased  budgets  to  specific  contract  tasks  or  statements 
of  work,  or  both; 

•  indicate  work  progress; 

•  properly  relate  cost,  schedule,  and  technical  accomplishments; 

•  are  valid,  timely,  and  auditable; 

•  provide  managers  with  summarized  information  at  a  practical 
level;  and 

•  are  derived  from  the  same  internal  EVMS  that  the  contractor  uses 
to  manage  the  contract. 

The  DoD  adopted  industry-standard  criteria  that  define  acceptable  requirements  to 
implement  EVMS  on  Defense  contracts. 

Estimate-at-Completion  Calculations 


Calculations  for  an  EVMS  are  based  on  three  key  data  points  in  the  contractor’s 
cost  accounting  system;  specifically,  the  budgeted  cost  of  work  scheduled,  the 
budgeted  cost  of  work  performed,  and  the  actual  cost  of  work  performed.  By 
comparing  those  three  data  points,  managers  can  determine  current  schedule  and 
cost  variances  and  can  use  that  data  to  predict  possible  schedule  delays  or  cost 
overruns.  Estimate-at-completion  (EAC)  calculations  are  used  to  predict  how 
much  a  contract  will  cost  if  current  schedule  and  cost  variances  continue.  The 
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Defense  Systems  Management  College  defines  EAC  as  the  aetual  cost  of  work 
performed  plus  an  estimate  for  remaining  work,  and  identifies  the  following 
standard  formulas  for  calculating  the  estimate  at  completion. 


1.  EACrPT  =  BAC 

CPI 

2.  EACcomposite  =  ACWPcuM  +  (BAC  -  BCWPcum) 


(CPIcUM  *  SPIcUM  ) 

ACWP  Actual  Cost  of  Work  Performed 

BAC  Budget  at  Completion 

BC  WP  Budgeted  Cost  of  Work  Performed 

CPI  Cost  Performance  Index 

CUM  Cumulative 

SPI  Schedule  Performance  Index 

EAC  Using  Formulas.  Because  the  program  office  completed  an  EVMS 
rephasing  effort  in  December  2001,  we  limited  our  review  of  contract  EVMS 
calculations  to  the  months  after  the  program  restructure.  Using  the  EVMS  figures 
provided  by  the  program  office,  we  calculated  the  EAC  for  the  current  ADS 
engineering  and  manufacturing  development  contract  using  each  of  the  formulas 
listed  above,  and  calculated  the  resulting  variance  or  estimated  cost  overruns.  The 
EAC  for  Eormula  1  includes  cost  variables  and  the  EAC  for  Eormula  2  includes 
both  cost  and  schedule  variables.  The  following  table  shows  the  results  of  the 
EAC  calculations  and  total  estimated  cost  overruns  for  the  contract  at  the  end  of 
each  month.  Appendix  C  provides  a  breakout  of  the  calculations  for  each 
authorized  work  element  within  the  contract  work  breakdown  structure. 

Month-End  Estimated  Cost  Overruns 

(in  millions) 

Eormula  One  Eormula  Two 


Month 

EAC 

Estimated 

Cost  Overruns 

EAC 

Estimated 
Cost  Overruns 

January 

$68.8 

$6.2 

$70.8 

$8.2 

Eebruary 

68.8 

6.2 

69.6 

7.1 

March 

69.2 

6.7 

72.1 

9.5 

April 

69.5 

7.0 

72.5 

9.9 

EAC  Using  the  Program  Office  Method.  The  program  office  used  standard 
formulas  to  calculate  cost  and  schedule  variances  and  cost  and  schedule 
performance  indices  based  on  EVMS  data.  However,  the  program  office  did  not 
use  standard  formulas  to  calculate  the  EAC,  which  is  an  indication  of  whether  a 
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program  will  be  completed  on  time  and  within  budget.  The  program  office  did 
not  use  the  standard  formulas  because  it  believed  that  the  standard  formulas  did 
not  accurately  reflect  the  unique  aspects  of  the  ADS  program.  Instead,  the 
program  office  stated  that  it  calculated  the  EAC  by  adding  the  actual  cost  of  work 
performed  to  an  engineering  estimate  of  work  remaining.  However,  the  program 
office  was  unable  to  provide  documentation  on  the  process  used  to  develop  the 
engineering  estimate.  It  appears  that  the  program  office  calculated  the  EAC  by 
adding  the  actual  cost  of  work  performed  to  the  budgeted  cost  of  work  remaining. 
A  comparison  of  EAC  using  this  method  and  the  engineering  estimate  that  the 
program  office  provided  for  April  2002  shows  only  a  .2  percent  difference  in  the 
EAC  amounts  —  $66.2  million  as  calculated  and  $66.3  million  as  provided  by  the 
program  office.  This  method  assumes  that  the  remainder  of  the  contract  will  be 
completed  on  time  and  within  budget,  an  invalid  assumption  when  based  on 
contractor  performance  through  April  2002.  The  program  office  calculated  the 
January,  Eebruary,  March,  and  April  EAC  as  $64.8,  $65,  $65.6,  and  $66.3 
million,  respectively,  resulting  in  estimated  cost  overruns  of  $0.1,  $0.3,  $0.9  and 
$1.6  million,  respectively.  In  making  its  EAC  calculations,  it  appears  that  the 
program  office  did  not  consider  the  contractor’s  past  performance  on  this  contract 
in  predicting  future  contract  performance.  The  ADS  program  manager 
recognized  that  the  EAC  was  unfavorable  but  assumed  that  savings  in  test  article 
production  costs  would  offset  cost  overruns  experienced  during  development; 
however,  future  savings  cannot  be  predicted  and  relied  upon. 

The  following  figure  shows  a  comparison  of  the  estimated  cost  overruns 
calculated  by  the  program  office  and  those  calculated  by  using  the  standard 
formulas  for  January  through  April  2002. 


1/3 

o 


a 


C/) 

a 


o 

Q 


January  Eebruary  March  April 


□  Program  Office 

□  Eormula  One 

□  Eormula  Two 


Estimated  Cost  Overruns  in  Millions 
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Defense  Contract  Management  Agency  Oversight 


Usually,  the  Defense  Contraet  Management  Agency  (DCMA)  reviews 
contractors’  cost  and  schedule  control  systems  for  acquisition  program  offices 
using  earned  value  management  techniques.  After  contract  award,  DCMA 
monitors  contractors’  performance  and  management  systems  to  ensure  that  cost, 
product  performance,  and  delivery  schedules  comply  with  the  terms  and 
conditions  of  the  contracts.  DCMA  also  provides  EVMS  tracking  services  to 
program  offices.  EVMS  tracking  services  include  validating  contractors’  EVMS, 
calculating  schedule  and  cost  variances,  notifying  program  management  of  areas 
of  concern,  and  tracking  those  areas  from  month  to  month. 

As  of  June  2002,  the  DCMA  and  the  ADS  program  office  had  not  approved  a 
memorandum  of  agreement  to  document  a  planned  certification,  review,  and 
reporting  process  for  the  ADS  engineering  and  manufacturing  development 
contract.  A  draft  memorandum  of  agreement  was  being  circulated  within  the 
program  office  and  DCMA  for  review  and  comment.  However,  the  program 
office,  in  the  draft  memorandum  of  agreement,  did  not  plan  to  request  DCMA 
support  to  perform  earned  value  management  calculations  and  limited  the  DCMA 
oversight  to  validating  the  contractor’s  EVMS.  The  program  manager  believed 
that  his  personnel  were  better  equipped  to  provide  oversight  of  the  contractor’s 
research  and  development  efforts.  In  the  draft  memorandum  of  agreement,  the 
program  office  retained  the  responsibility  for  calculating  schedule  and  cost 
variances  and  EAC. 

When  asked,  the  DCMA  program  integrator  for  ADS  did  not  agree  with  the 
program  office’s  assumptions  or  methodology  for  calculating  the  contract  EAC. 
He  stated  that  he  preferred  to  use  standard  Eormula  2  to  calculate  EAC  because  it 
considered  both  cost  and  schedule  variables.  However,  he  cautioned  that  while 
there  is  no  one  formula  that  considers  all  variables,  the  formula  used  should  be 
considered  a  “tool”  to  flag  areas  of  concern  needing  further  explanation  or 
mitigation.  Based  on  the  April  2002  EVMS  data,  calculations  using  Eormula  2 
resulted  in  an  estimated  cost  overrun  of  $9.9  million,  but  the  program  office 
estimated  a  cost  overrun  of  $1.6  million.  The  $8.3  million  difference  is  more  than 
10  percent  of  the  total  contract  value. 

Conclusion 


The  ADS  Program  Office’s  calculation  of  EAC  did  not  provide  accurate  EVMS 
data  to  effectively  manage  program  cost  and  performance,  and  did  not  provide 
acquisition  decision  makers  with  the  information  needed  to  make  fully  informed 
investment  decisions.  EVMS  calculations  are  the  primary  means  of  providing  the 
program  manager  and  decision  makers  with  contractor  performance  information. 
Without  accurate  EVMS  data,  the  program  office  cannot  monitor  contract 
progress  and  manage  the  program  effectively.  Although  several  formulas  may  be 
used  to  calculate  EAC  and  many  factors  should  be  considered,  the  lack  of 
documentation  supporting  the  program  office  methodology  and  the  results 
obtained  from  using  either  of  the  standard  formulas  call  that  methodology  into 
question. 
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Recommendation  and  Management  Comments 


B.  We  recommend  that  the  Program  Manager,  Advanced  Deployable 
System  revise  the  current  draft  memorandum  of  agreement  between  the 
program  office  and  the  Defense  Contract  Management  Agency  to  include 
Defense  Contract  Management  Agency  oversight  of  contractor  earned  value 
management  calculations,  including  standard  estimate-at-completion 
formulas. 

Management  Comments.  The  Deputy  Assistant  Secretary  of  the  Navy 
(Planning,  Programming  and  Resources)  concurred,  stating  that  the  program 
office  and  the  Space  and  Naval  Warfare  Command  will  develop  a  memorandum 
of  agreement  with  the  Defense  Contract  Management  Agency  no  later  than 
November  1,  2002.  He  stated  that  all  future  earned  value  reports  will  include 
estimate-at-completion  values  calculated  using  standard  formulas  along  with  the 
program  manager’s  estimate-at-completion. 


10 


C.  Program  Documentation 

The  ADS  Program  Office  had  not  completed  actions  to  update  the 
acquisition  strategy;  cost  analysis  requirements  description;  life-cycle  cost 
estimate;  support  plan;  and  programmatic  environmental,  safety,  and 
health  evaluation  plan  as  required  to  show  the  current  status  of  the 
program.  This  condition  occurred  because  the  program  office  could  not 
update  program  documentation  because  an  updated  ORD  and  test  and 
evaluation  master  plan  were  not  yet  approved.  As  a  result,  the  program 
manager  did  not  have  up-to-date  acquisition  documentation  needed  to 
effectively  manage  program  cost  and  performance  and  acquisition 
decision  makers  could  not  make  fully  informed  investment  decisions. 

Program  Documentation  Requirements 


DoD  Policy.  DoD  Instruction  5000.2  and  DoD  Regulation  5000. 2-R,  “Mandatory 
Procedures  for  Major  Defense  Acquisition  Programs  (MDAPs)  and  Major 
Automated  Information  System  (MAIS)  Acquisition  Programs,”  April  2002,^ 
establish  policies  and  procedures  for  managing  acquisition  programs.  The 
DoD  5000  documents  state  that  program  managers  for  defense  acquisitions  are  to 
rely  on  and  generate  program  documents  needed  for  program  execution  and 
decision  making.  Program  documents  include  the  acquisition  strategy'  cost 
analysis  requirements  description  (CARD);  life-cycle  cost  estimate;  CA  support 
plan;  programmatic  environmental,  safety,  and  health  evaluation  plan;  and 
acquisition  decision  memorandum.  Those  interrelated  documents  help  the 
program  manager  to  provide  decision  makers  with  the  information  needed  to 
oversee  and  make  important  program  decisions. 

Navy  Policy.  Secretary  of  the  Navy  Instruction  5000.2B,  “Implementation  of 
Mandatory  Procedures  for  Major  and  Non-Major  Defense  Acquisition  Programs 
and  Major  and  Non-Major  Information  Technology  Acquisition  Programs,” 
December  6,  1996,  provides  mandatory  procedures  for  Department  of  the  Navy 
implementation  of  DoD  Regulation  5 000. 2-R. 

Status  of  Program  Documents 


The  program  office  had  not  completed  actions  to  update  the  acquisition  strategy; 
CARD;  life-cycle  cost  estimate;  C"^I  support  plan;  and  programmatic 
environmental,  safety,  and  health  evaluation  plan  as  required  to  show  the  current 
status  of  the  program. 

Acquisition  Strategy.  DoD  Regulation  5 000. 2-R  requires  the  program  manager 
to  develop  the  acquisition  strategy  at  program  initiation  and  update  it  whenever 
there  is  a  change  or  as  the  system  approach  and  program  elements  are  better 
defined.  The  program  manager  develops  and  documents  an  acquisition  strategy  to 
serve  as  a  road  map  from  program  initiation  through  post-production  support. 


^Prior  versions  of  DoD  Regulation  5000. 2-R  also  eontain  the  requirements  outlined  in  this  report. 
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The  program  office  prepared  the  initial  acquisition  strategy  in  April  1994  in 
preparation  for  program  initiation.  It  was  revised  in  January  2000  for  the 
engineering  and  manufacturing  review.  The  program  manager  stated  that  the 
acquisition  strategy  was  being  updated  again  because  of  a  decision  to  purchase 
specific  equipment  using  a  sole-source  contract,  rather  than  using  a  competitive 
contracting  process  identified  in  the  original  acquisition  strategy.  In  addition  to 
the  change  in  contracting  procedures,  the  acquisition  strategy  needed  to  be 
updated  to  show  the  four-block  evolutionary  acquisition  strategy  adopted  by  the 
program  office.  The  program  office  developed  an  acquisition  strategy  outline  in 
May  2002,  and  planned  to  revise  it  by  August  2002.  The  revised  acquisition 
strategy  will  be  completed  in  time  to  support  the  program  review  that  the 
Assistant  Secretary  of  the  Navy  (Research,  Development,  and  Acquisition) 
requested  in  the  May  25,  2001,  update  to  the  acquisition  decision  memorandum 
that  approved  the  program  restructure.  The  review  was  to  take  place  1  year  after 
the  program  restructure  was  approved  and  is  currently  scheduled  for  the  fourth 
quarter  of  FY  2002. 

Cost  Analysis  Requirements  Description.  DoD  Regulation  5000. 2-R  requires 
DoD  Components  for  acquisition  category  I  programs  to  establish,  as  a  basis  for 
the  life-cycle  cost  estimate,  a  description  of  the  salient  features  of  the  acquisition 
program  and  of  the  system  itself  The  description,  referred  to  as  the  CARD,  is  to 
be  flexible,  tailored,  and  refer  to  information  available  in  other  documents. 
Secretary  of  the  Navy  Instruction  5000.2B  extends  the  CARD  requirement  to  all 
acquisition  programs.  DoD  Manual  5000.4-M,  “Department  of  Defense  Cost 
Analysis  Guidance  and  Procedures,”  December  11,  1992,  provides  guidance  on 
preparing  the  CARD.  The  Manual  requires  that  the  CARD  be  prepared  by  the 
program  office  and  approved  by  the  DoD  Component's  Program  Executive 
Officer.  The  CARD  is  to  contain  the  following  elements:  a  system  overview, 
system  risk  assessment,  system  operational  concept,  quantity  requirements, 
system  manpower  requirements,  system  activity  rates,  a  milestone  schedule,  an 
acquisition  strategy,  a  development  plan,  facility  requirements,  and  the 
contractor’s  cost  data  reporting  plan.  Additionally,  the  Manual  requires  that  the 
CARD  be  regarded  as  a  “living”  document  that  is  updated  to  show  any  changes 
that  have  occurred  or  the  availability  of  new  data. 

The  program  office  prepared  the  CARD  for  the  ADS  program  in  October  1999. 
The  CARD  was  divided  into  sections,  each  section  corresponding  to  the  required 
elements  as  stated  above.  The  system  overview  provided  a  description  of  the 
ADS  and  mentioned  the  two  deployment  options.  Platform  Alpha  and  Platform 
Bravo.  However,  because  the  original  plan  was  to  produce  Platform  Alpha  before 
Platform  Bravo,  the  CARD  discussed  only  Platform  Alpha  in  the  remaining 
sections  and  stated  that  a  new  CARD  would  be  produced  for  Platform  Bravo.  As 
of  June  2002,  the  Program  Office  had  not  updated  the  CARD  to  discuss  the 
unique  aspects  of  Platform  Bravo  even  though  the  acquisition  plan  for  the  ADS 
involves  producing  and  deploying  Platform  Bravo  before  Platform  Alpha. 
Additionally,  the  sections  on  system  risk  and  system  operational  concepts  were 
incomplete  and  referred  the  reader  to  undefined  documents  for  information. 
Accordingly,  the  program  office  needs  to  update  and  complete  the  CARD  so  that 
cost  estimators  can  use  it  to  develop  a  reliable  life-cycle  cost  estimate  for  the 
program. 
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The  program  office  stated  that  it  was  preparing  updates  to  the  CARD  and  that 
each  block  would  have  an  appendix  within  the  CARD  that  described  the  unique 
aspects  of  that  block.  However,  the  program  office  did  not  plan  to  prepare  the 
appendix  for  each  block  until  the  program  office  completes  a  detailed  design 
review  for  each  block.  As  a  consequence,  cost  estimators  will  not  be  able  to 
provide  the  program  office  with  reliable  total  cost  estimates  until  late  in  the  life 
cycle  of  the  program,  which,  in  turn,  will  prevent  decision  makers  from  making 
early  investment  decisions  that  are  based  on  total  program  costs. 

Life-Cycle  Cost  Estimate.  The  life-cycle  cost  estimate  represents  the  total  cost 
to  the  Government  for  the  acquisition  and  ownership  of  a  system  over  its  useful 
life.  It  includes  the  cost  of  development,  acquisition,  operations  and  support  and, 
where  applicable,  disposal.  DoD  Regulation  5000. 2-R  requires  that  program 
managers  for  all  major  defense  acquisition  programs  prepare  a  life-cycle  cost 
estimate  in  support  of  program  initiation  and  at  all  subsequent  milestone  reviews. 
Secretary  of  the  Navy  Instruction  5000. 2B  extends  the  life-cycle  cost  estimate 
requirement  to  all  acquisition  programs. 

The  program  office  prepared  a  life-cycle  cost  estimate  for  the  engineering  and 
manufacturing  program  review  in  November  1999.  The  program  office  based  the 
life-cycle  cost  estimate  on  the  assumption  that  Platform  Alpha  would  be 
developed  and  procured  before  Platform  Bravo.  However,  in  April  2001,  the 
Office  of  the  Chief  of  Naval  Operations  directed  the  program  office  to  develop 
and  procure  Platform  Bravo  first  and  identified  deployment  options  for  each 
platform.  In  response,  the  program  office  initiated  a  four-block  acquisition 
strategy.  As  of  June  2002,  the  program  office  had  not  determined  the  additional 
time  and  cost  associated  with  the  new  acquisition  strategy  and  had  not  calculated 
disposal  costs.  Many  of  the  decisions  throughout  the  life  of  the  program  are 
based  on  the  life-cycle  costs,  not  the  least  of  which  is  to  determine  the  appropriate 
level  of  program  oversight.  Program  oversight  is  dependent  on  estimated 
program  costs  for  research,  development,  test,  and  evaluation  and  procurement. 
Further,  without  a  realistic  life-cycle  cost  estimate,  program  decision  makers  will 
not  be  able  to  oversee  and  evaluate  program  management  and  affordability. 

The  program  office  stated  that  separate  life-cycle  cost  estimates,  to  include 
disposal  costs,  will  be  prepared  for  each  ADS  block  as  it  becomes  defined  in 
sufficient  detail.  The  Block  I  life-cycle  cost  estimate  and  the  preliminary  life- 
cycle  cost  estimate  for  Block  II  will  be  available  for  the  fourth  quarter  FY  2002, 
Assistant  Secretary  of  the  Navy  (Research,  Development,  and  Acquisition) 
program  review.  The  life-cycle  cost  estimate  for  Block  II  will  include 
development  and  production  costs,  but  not  operation  and  support  costs  because 
those  costs  will  not  be  sufficiently  defined.  The  program  office  plans  to  have 
preliminary  life-cycle  cost  estimates  for  Blocks  III  and  IV  in  FY  2003.  The 
program  office  stated  that  complete  life-cycle  cost  estimates  for  Blocks  II  through 
IV,  including  operation  and  support  costs  and  disposal  costs,  will  not  be  available 
until  after  the  system  design  review  for  each  block  and  will  be  updated  after  the 
detailed  design  review  for  each  block.  While  this  schedule  shows  a  plan  for 
updating  the  life-cycle  cost  estimate,  the  program  office  will  not  have  a 
reasonable  estimate  of  total  program  cost  until  late  in  the  engineering  and 
manufacturing  development  phase  of  the  acquisition  process.  To  make 
affordability  decisions  for  the  ADS  program,  the  program  office  needs  to 
complete  a  comprehensive  life-cycle  cost  estimate  to  include  all  costs  of  the 
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four-block  program,  even  if  costs  for  Blocks  II  through  IV  cannot  be  completed  in 
detail.  The  program  office  can  update  the  life-eyele  cost  estimate  as  neeessary, 
but  the  program  office  needs  to  have  an  estimate  of  total  life-eyele  costs  now  to 
make  management  and  affordability  decisions  for  the  program.  Additionally,  an 
independent  life-eyele  cost  estimate  was  not  performed  earlier  beeause  an 
independent  estimate  is  not  required  for  acquisition  category  II  programs.  When 
the  program  is  redesignated  as  an  acquisition  category  I  program  as  required,  an 
independent  life-cycle  cost  estimate  must  be  prepared. 

C"*!  Support  Plan,  DoD  Regulation  5000. 2-R  requires  that  DoD  Components 
develop  C"^I  support  plans  for  all  programs  early  in  the  aequisition  proeess  when 
the  Components  connect  in  any  way  to  the  communication  and  information 
infrastructure.  Additionally,  DoD  Instruction  5000.2  requires  eaeh  program 
manager  to  address  system  interoperability  in  the  C"^I  support  plan.  The 
Instruction  defines  interoperability  as  the  ability  of  systems,  units,  or  forees  to 
provide  services  to  or  accept  services  from  other  systems,  units,  or  forces  and  to 
use  the  services  so  exehanged  to  operate  effectively  together.  The  Instruction  also 
states  that  the  outcome  of  systems  aequisition  is  a  system  that  is  interoperable 
with  other  systems  (U.S.,  Coalition,  and  allied  systems,  as  speeified  in  the  ORD). 
Further,  aequisition  decision  makers  are  required  to  review  the  C"^I  support  plan  at 
eaeh  milestone;  at  deeision  reviews,  as  appropriate;  and  whenever  support 
requirements  change. 

The  program  office  reeognized  the  need  for  a  C"^I  support  plan  based  on  ADS 
interfaces  with  other  C"^I  systems,  and  prepared  the  initial  C"^I  support  plan  during 
the  program  definition  and  risk  reduction  phase  of  the  acquisition  process.  The 
program  offiee  plans  to  update  its  C"^!  support  plan  in  FY  2004  beeause  the  Spaee 
and  Naval  Warfare  Systems  Command  issued  guidanee  requiring  that  all 
Command  programs  set  aside  FY  2004  funds  to  update  C^^I  support  plans.  In 
addition  to  the  availability  of  funding,  program  offiee  personnel  believed  that 
collaborating  with  other  program  offices  that  were  updating  their  C"^I  support 
plans  in  FY  2004  would  result  in  a  better  C"^I  support  plan. 

However,  an  important  funetion  of  a  C"^I  support  plan  is  to  identify  interfaces  that 
need  to  be  tested.  The  first  land  integration  test  where  ADS  interfaees  will  be 
tested  is  scheduled  for  the  first  quarter  FY  2004.  Aceordingly,  it  is  too  late  in  the 
proeess  for  the  program  offiee  to  wait  until  FY  2004  to  begin  updating  the  C"^I 
support  plan  to  incorporate  the  updated  operational  requirements  and  system 
configuration  changes.  Personnel  from  the  Office  of  the  Commander,  Operational 
Test  and  Evaluation  Force  stated  that  the  C"^I  support  plan  should  be  completed 
before  the  land  integration  test. 

Programmatic  Environmental,  Safety,  and  Health  Evaluation  Plan.  DoD 

Regulation  5000.2-R  requires  that  the  program  manager  include  a  programmatie 
environmental,  safety,  and  health  evaluation  plan  in  the  aequisition  strategy.  The 
program  manager  is  to  initiate  the  evaluation  at  the  earliest  possible  time  in 
support  of  a  program  initiation  deeision  and  maintain  an  updated  evaluation 
throughout  the  life  cyele  of  the  program.  The  program  manager  is  to  ensure  that 
the  system  can  be  tested,  operated,  maintained,  repaired,  and  disposed  of  in 
complianee  with  environmental  regulations  and  DoD  Regulation  5000.2-R 
requirements. 
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The  Naval  Facilities  Engineering  Service  Center  prepared  a  programmatic 
environmental,  safety,  and  health  evaluation  plan  for  ADS  in  May  1997,  and 
updated  it  in  July  1999.  The  Center  identified  potential  risks  that  could  occur 
despite  program  office  compliance  and  mitigation  activities.  Moderate  risks 
identified  included  concerns  involving  marine  mammals  and  endangered  species, 
personnel  safety  hazards  because  of  the  lithium  battery  storage  and  use,  and  air 
emissions  that  may  become  an  issue  as  production  rates  increase. 

One  mitigation  method  that  the  program  office  used  during  the  tests  was  marine 
mammal  watching  as  they  gradually  increased  acoustic  sound  to  the  desired 
testing  level.  Also,  the  primary  hardware  manufacturer  prepared  a  preliminary 
point  paper  primarily  addressing  disposal  of  batteries  and  cables.  When  the 
programmatic  environmental,  safety,  and  health  evaluation  plan  was  completed, 
the  program  office  identified  no  safety  hazards.  In  addition,  two  environmental 
assessments  were  conducted;  both  resulted  in  findings  of  no  significant  impact. 
However,  the  change  in  the  deployment  method  of  ADS  may  affect  the 
environment  and  the  safety  and  health  of  the  crew.  As  a  result,  the  program  office 
stated  that  the  Naval  Facilities  Engineering  Service  Center  was  updating  the 
programmatic  environmental,  safety,  and  health  evaluation  plan  concurrently  with 
the  ORD  approval  process. 

Conclusion 


Documentation  is  the  primary  means  of  providing  the  milestone  decision 
authority,  as  well  as  other  key  managers,  with  information  needed  for  decision 
making.  Without  accurate  and  up-to-date  program  documents,  such  as  the 
acquisition  strategy,  CARD,  life-cycle  cost  estimate,  C"^I  support  plan,  and 
programmatic  environmental,  safety,  and  health  plan,  the  program  office  cannot 
provide  assurance  to  acquisition  decision  makers  that  performance  and  cost 
thresholds  are  being  achieved  and  that  the  program  is  affordable.  Although  the 
program  office  believed  that  the  ORD  would  be  approved  in  August  2002,  with 
the  test  and  evaluation  master  plan  being  approved  shortly  thereafter,  the  ADS 
program  office  did  not  have  the  up-to-date  acquisition  documentation  needed  to 
effectively  manage  program  cost  and  performance,  and  acquisition  decision 
makers  could  not  make  fully  informed  investment  decisions. 

Recommendations  and  Management  Comments 


C,  We  recommend  that  the  program  manager  for  the  Advanced  Deployable 
System,  concurrent  with  the  approval  of  the  operational  requirements 
document  and  the  test  and  evaluation  master  plan: 

1,  Update  the  acquisition  strategy  to  incorporate  the  restructured  four- 
hlock  evolutionary  acquisition  strategy. 

2.  Update  the  cost  analysis  requirements  description  to  he  used  as  a 
basis  for  the  life-cycle  cost  estimate. 
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3.  Update  the  life-cycle  cost  estimate,  including  disposal  costs,  for  all 
four  blocks  of  the  evolutionary  acquisition  strategy  and  direct  the 
independent  review  of  the  life-cycle  cost  estimate, 

4.  Update  the  command,  control,  communications,  computers,  and 
intelligence  support  plan  to  document  required  interfaces  with  other  systems, 

5.  Update  the  programmatic  environmental,  safety,  and  health 
evaluation  plan  to  show  that  the  system  can  be  tested,  operated,  maintained, 
repaired,  and  disposed  of  in  compliance  with  environmental  regulations. 

Management  Comments,  The  Deputy  Assistant  Secretary  of  the  Navy 
(Planning,  Programming  and  Resources)  concurred  and  provided  the  following 
dates  for  completing  the  document  revisions: 

•  Updates  to  the  acquisition  strategy  and  life-cycle  cost  estimate,  inclusive 
of  all  four  blocks  of  the  Advanced  Deployable  System  program,  will  be 
completed  no  later  than  February  28,  2003. 

•  Updates  to  the  command,  control,  communications,  computers,  and 
intelligence  support  plan  and  programmatic  environmental,  safety,  and 
health  evaluation  plan  will  be  completed  no  later  than  October  30,  2002. 

Additionally,  the  program  office  is  developing  a  plan  of  action  and  milestones  to 
drive  and  track  the  process  for  updating  all  needed  documentation  in  a  timely 
manner.  This  action  plan  will  be  completed  before  the  program  review  planned 
for  October  2002. 
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Appendix  A.  Scope  and  Methodology 


In  conducting  this  program  audit,  we  reviewed  doeumentation  dated  from  Mareh 
1993  through  May  2002.  We  interviewed  and  obtained  doeumentation  from  the 
staffs  of  the  Direetor,  Operational  Test  and  Evaluation;  Direetor,  Defense 
Contraet  Management  Agency;  Assistant  Secretary  of  the  Navy  (Researeh, 
Development,  and  Aequisition);  Deputy  Chief  of  Naval  Operations  (Warfare 
Requirements  and  Programs);  Commander,  Operational  Test  and  Evaluation 
Eoree;  Commander,  Spaee  and  Naval  Warfare  Command;  and  Program  Manager, 
Advaneed  Deployable  System.  We  used  eriteria  in  DoD  Regulation  5000.2-R  to 
perform  the  audit.  To  accomplish  the  audit  objectives,  we  took  the  following 
steps: 

•  determined  whether  the  users  had  adequately  defined  the  system 
requirements; 

•  determined  whether  the  program  office  had  developed  and  implemented 
an  aequisition  plan,  a  risk  management  plan,  and  a  test  and  evaluation 
plan; 

•  evaluated  the  Defense  Contract  Management  Ageney’s  involvement  in 
monitoring  the  contractor’s  earned  value  management  proeess; 

•  evaluated  the  program  office’s  management  of  contracts  for  the  program; 

•  determined  whether  the  program  office  had  a  fully  developed, 
programmatic,  environmental,  safety,  and  health  evaluation  plan; 

•  assessed  the  program  offiee’s  implementation  of  the  DoD  environmental 
management  proeess; 

•  determined  whether  the  program  office  had  prepared  a  life-cycle  cost 
estimate  for  the  program; 

•  evaluated  program  office  use  of  integrated  produet  teams;  and 

•  reviewed  management  eontrols  related  to  the  audit  objeetive. 

Audit  Dates  and  Standards.  We  performed  this  audit  from  October  2001 
through  June  2002  in  accordance  with  generally  accepted  government  auditing 
standards. 

Use  of  Computer-Processed  Data,  We  did  not  use  computer-processed  data  to 
perform  this  audit. 

Contacts  During  the  Audit.  We  visited  or  contacted  individuals  and 
organizations  within  the  DoD  and  eontraetor  loeations. 
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General  Accounting  Office  High-Risk  Area,  The  General  Aeeounting  Offiee 
has  identified  several  high-risk  areas  in  the  DoD.  This  report  provides  eoverage 
of  the  DoD  Weapons  System  Aequisition  high-risk  area. 


Management  Control  Program  Review 


DoD  Direetive  5010.38,  “Management  Control  (MC)  Program,”  August  26,  1996, 
and  DoD  Instruetion  5010.40,  “Management  Control  (MC)  Program  Proeedures,” 
August  28,  1996,  require  DoD  organizations  to  implement  a  eomprehensive 
system  of  management  eontrols  that  provides  reasonable  assuranee  that  programs 
are  operating  as  intended  and  to  evaluate  the  adequaey  of  the  eontrols. 

Scope  of  the  Review  of  the  Management  Control  Program.  In  accordance 
with  DoD  Regulation  5000.2-R,  acquisition  managers  are  to  use  program  cost, 
schedule,  and  performance  parameters  as  control  objectives  to  implement  the 
requirements  of  DoD  Directive  5010.38.  Accordingly,  we  limited  our  review  to 
management  controls  directly  related  to  program  definition,  structure,  design 
assessments  and  decision  reviews,  and  periodic  reporting. 

Adequacy  of  Management  Controls,  We  identified  material  management 
control  weaknesses  for  the  acquisition  of  the  ADS  as  defined  by  DoD 
Instruction  5010.40.  Controls  over  program  documentation  were  insufficient  to 
ensure  that  acquisition  decision  makers  were  provided  updated  acquisition 
documentation  in  time  to  effectively  manage  program  cost  and  performance. 
Recommendations  C.I.,  C.2.,  C.3.,  C.4.,  and  C.5.,  if  implemented,  will  improve 
the  quality  and  timeliness  of  program  information  provided  to  acquisition  decision 
makers  so  they  can  make  fully  informed  investment  decisions.  A  copy  of  the 
report  will  be  provided  to  the  senior  official  responsible  for  management  controls 
in  the  Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics  and  the  Department  of  the  Navy. 

Adequacy  of  Management’s  Self  Evaluation.  In  June  1998,  the  Space  and 
Naval  Warfare  Command  identified  the  ADS  program  office  as  an  assessable 
unit.  However,  in  that  evaluation,  the  Space  and  Naval  Warfare  Command  did 
not  identify  the  specific  material  management  control  identified  by  the  audit 
because  the  weakness  occurred  after  the  June  1998  evaluation. 

Prior  Coverage 


During  the  last  5  years,  the  General  Accounting  Office,  the  Inspector  General  of 
the  Department  of  Defense,  and  the  Naval  Audit  Service  have  not  issued  reports 
specifically  addressing  the  ADS. 
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Appendix  B.  Definitions  of  Technical  Terms 


Acquisition  Phase.  An  acquisition  phase  represents  all  the  tasks  and  activities 
needed  to  bring  a  program  to  the  next  major  milestone.  Phases  provide  a  logical 
means  of  progressively  translating  broadly  stated  mission  needs  into  well-defined, 
system-specific  requirements  and,  ultimately,  into  operationally  effective, 
suitable,  and  survivable  systems. 

Evolutionary  Acquisition.  An  evolutionary  aequisition  is  an  aequisition  strategy 
that  defines,  develops,  produees  or  acquires,  and  fields  an  initial  hardware  or 
software  inerement  of  operational  capability.  There  are  two  basic  approaches  to 
evolutionary  aequisition.  In  one  approaeh,  the  ultimate  purpose  can  be  defined  at 
the  beginning  of  the  program,  with  the  content  of  each  deployable  increment 
determined  by  the  maturation  of  key  technologies.  In  the  seeond  approach,  the 
ultimate  purpose  eannot  be  defined  at  the  beginning  of  the  program,  and  eaeh 
increment  of  the  capability  is  defined  by  the  maturation  of  the  teehnologies 
matehed  with  the  evolving  needs  of  the  user. 

Engineering  and  Manufacturing  Development.  Engineering  and 
manufaeturing  development  is  the  third  phase  of  the  aequisition  process  where  the 
program  office  and  its  contractors  fully  develop,  engineer,  design,  fabricate,  test, 
and  evaluate  the  systems  and  the  principal  items  neeessary  for  its  support. 

Full-Rate  Production.  Full-rate  production  is  contracting  for  economic 
production  quantities  following  stabilization  of  the  system  design  and  validation 
of  the  produetion  process. 

Milestone.  A  milestone  is  the  point  where  the  milestone  decision  authority 
decides  whether  to  start  or  eontinue  an  aequisition  program  in  the  aequisition 
proeess. 

Milestone  Decision  Authority.  A  milestone  deeision  authority  is  the  individual 
designated  in  accordanee  with  criteria  established  by  the  Under  Secretary  of 
Defense  for  Aequisition,  Teehnology,  and  Logistics  to  approve  entry  of  an 
aequisition  program  in  to  the  next  phase  of  the  aequisition  proeess. 

Operational  Requirements  Document.  The  ORD  is  a  formatted  statement 
containing  performance  and  related  operational  performance  parameters  for  the 
proposed  concept  or  system.  The  ORD  is  prepared  by  the  user  or  user’s 
representative. 

Programmatic  Environmental,  Safety,  and  Health  Evaluation  Plan.  A 

programmatic  environmental,  safety,  and  health  evaluation  plan  is  written  to 
doeument  the  program  manager’s  strategy  for  meeting  environmental,  safety,  and 
health  requirements;  establish  responsibilities;  and  identify  how  progress  will  be 
tracked. 
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Test  and  Evaluation  Master  Plan,  The  test  and  evaluation  master  plan 
doeuments  the  overall  structure  and  objectives  of  the  test  and  evaluation  program. 
It  provides  a  framework  within  which  to  generate  detailed  test  and  evaluation 
plans  and  documents  schedule  and  resource  implications  associated  with  the  test 
and  evaluation  program.  The  test  and  evaluation  master  plan  identifies  the 
necessary  developmental  test  and  evaluation,  operational  test  and  evaluation,  and 
live-fire  test  and  evaluation  activities. 
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Budgeted  Cost  of  Work  Scheduled  *Cost  Performance  Index  (BCWP  /  ACWP) 

Budgeted  Cost  of  Work  Performed  ‘Budget  at  Completion 

Actual  Cos  of  Work  Performed  ^Estimate  at  Completion  (Budget  /  CPI) 

Schedule  Performance  Index  (BCWP/BCWS)  Variance  at  Completion  (BAC  -  EAC) 

’Estimate  at  Completion  ((BAC  '  BCWP)  /  (CPI  *  SPI)  +  ACWP) 


Appendix  D.  Report  Distribution 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  (Acquisition,  Technology  and  Logistics) 
Under  Secretary  of  Defense  (Comp troller)/Chief  Financial  Officer 
Deputy  Chief  Financial  Officer 
Deputy  Comptroller  (Program/Budget) 


Department  of  the  Army 

Auditor  General,  Department  of  the  Army 


Department  of  the  Navy 

Assistant  Secretary  of  the  Navy  (Financial  Management  and  Comptroller) 
Assistant  Secretary  of  the  Navy  (Manpower  and  Reserve  Affairs) 

Assistant  Secretary  of  the  Navy  (Research,  Development,  and  Acquisition) 

Naval  Inspector  General 

Auditor  General,  Department  of  the  Navy 

Deputy  Chief  of  Naval  Operations  (Warfare  Requirements  and  Programs) 
Commander,  Space  and  Naval  Warfare  Systems  Command 

Program  Director,  Intelligence,  Surveillance,  and  Reconnaissance  Systems 
Program  Manager,  Advanced  Deployable  System 


Department  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 
Auditor  General,  Department  of  the  Air  Force 


Non-Defense  Federal  Organization 

Office  of  Management  and  Budget 
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Congressional  Committees  and  Subcommittees,  Chairman  and 
Ranking  Minority  Member 

Senate  Committee  on  Appropriations 

Senate  Subeommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Serviees 
Senate  Committee  on  Governmental  Affairs 
House  Committee  on  Appropriations 

House  Subeommittee  on  Defense,  Committee  on  Appropriations 
House  Committee  on  Armed  Serviees 
House  Committee  on  Government  Reform 

House  Subeommittee  on  Government  Effleieney,  Finaneial  Management,  and 
Intergovernmental  Relations,  Committee  on  Government  Reform 
House  Subeommittee  on  National  Seeurity,  Veterans  Affairs,  and  International  Relations, 
Committee  on  Government  Reform 

House  Subeommittee  on  Teehnology  and  Proeurement  Poliey,  Committee  on 
Government  Reform 


26 


Department  of  the  Navy  Comments 


DEPARTMENT  OF  THE  NAVY 
OFFICE  OF  THE  ASSISTANT  SECRETARY 
RESEARCH.  DEVELOPMENT  AND  ACQUISITION 
1000  NAVY  PENTAQON 
WASHINOTON,  DC  SaSSS-IOOO 

il«  SEP  MK 


MEMORANDUM  FOR  ASSISTANT  INSPECTOR  GENERAL  FOR  AUDITING, 
DEPARTMENT  OF  DEFENSE 

Subj ;  DOD IG  AUDIT  PROPOSED  REPORT  “ACQUISITION  OF  THE  ADVANCED 
DEPLOYABLE  SYSTEM”  (D2002AE-003) 

Ref:  (a)  DoDIG  Audit  Report  on  the  Acquisition  of  the  Advanced  Deployable  System 

EncI;  (1)  DoN  Response  to  DodIG  Audit  Proposed  Report  Project  D2002-0003. 

In  response  to  reference  (a),  enclosure  (1)  is  forwarded  as  the  Department  of  the  Navy 
response  to  (he  subject  audit. 


William  J.  Schaefer 

Deputy  Assistant  Secretary  of  the  Navy 
Planning,  Programming  and  Resources 

Copy  to: 

NAVAL  INSPECTOR  GENERAL  (N43) 

CNO  (N77) 

SPA  WAR  (PMW  183)  (PD  18) 


DEPARTMENT  OF  THE  NAVY  RESPONSE 
TO 

DODIG  PROPOSED  REPORT  ON  ACQUISITION  OF  THE  ADVANCED  DEPLOYABLE 

SYSTEM 

(D2002AE-003) 


Finding  A:  The  ADS  program  manager  did  not  inform  the  Assistant  Secretary  of  the  Navy 
(Research,  Development,  and  Acquisition)  that  the  oversight  of  the  program  should  be  raised 
to  the  level  of  an  acquisition  category  I  program.  This  condition  occurred  because  the 
program  manager  believed  that  the  cost  information  showing  that  the  program  had  exceeded 
established  thresholds  for  research,  development,  test,  and  evaluation  had  been  provided  to 
the  Assistant  Secretary  of  the  Navy  and  that  no  further  action  on  his  part  was  required.  As  a 
result,  acquisition  management  oversight  was  not  provided  commensurate  with  that  required 
for  an  acquisition  category  I  program. 

Recommendation  A:  We  recommend  that  the  Assistant  Secretary  of  the  Navy  (Research, 
Development  and  Acquisition)  request  that  the  Under  Secretary  of  Defense  (Acquisition. 
Technology,  and  Logistics)  designate  the  Advanced  Deployable  System  as  an  acquisition  1 
program. 

DoN  Response:  Concurrence  pending  program  review.  The  Assistant  Secretary  of  the  Navy 
(Research,  Development  and  Acquisition)  has  requested  a  program  review  to  be  conducted 
with  the  Program  Executive  Officer  and  the  Program  Manager  prior  to  the  3 1  October.  Upon 
completion  of  the  review,  the  Department  of  the  Navy  vidll  make  a  determination  of 
appropriate  Acquisition  Category  for  the  ADS  program.  If  warranted,  the  Assistant  Secretary 
of  the  Navy  (Research,  Development  and  Acquisition  will  forward  a  request  to  the  Under 
Secretary  of  Defense  (Acquisition,  Technology,  and  Logistics)  to  designate  the  Advanced 
Deployable  System  program  as  an  acquisition  1  program. 

Finding  B:  The  program  office  did  not  apply  standard  estimate-at-completion  formulas  in 
calculating  earned  value  management  computations  because  it  did  not  believe  that  the 
standard  formulas  accurately  reflected  the  unique  aspects  of  the  ADS  program.  Also,  the 
program  office  did  not  request  the  Defense  Contract  Management  Agency  to  review  earned 
value  management  calculations  because  it  believed  its  personnel  were  better  equipped  to 
provide  oversight  of  the  research  and  development.  As  a  result,  the  program  office’s  EAC 
was  significantly  lower  than  the  EAC  calculated  using  standard  formulas  and  did  not  provide 
decision  makers  with  accurate  data  on  estimated  contract  cost  overruns. 

Recommendation  B:  We  recommend  that  the  Program  Manager,  Advanced  Deployable 
System  revise  the  cunent  draft  memorandum  of  agreement  between  the  program  office  and 
the  Defense  Contract  Management  Agency  to  include  Defense  Contract  Management 
Agency  oversight  of  contractor  earned  value  management  calculations,  including  standard 
estimate-at-completion  formulas. 


Enclosure  (1) 


28 


DoN  Response:  Concur.  The  program  oflfice  and  SPAWAR  will  develop  a  memorandum  of 
agreement  with  the  Defense  Contract  Management  Agency  no  later  than  0 1  November  2002. 
The  draft  memorandum  of  agreement  will  be  reviewed  and  approved  by  the  Assistant 
Secretary  of  (he  Navy,  Research,  Development  and  Acquisition,  prior  to  implementation.  All 
earned  value  reports  will  include  EAC  values  derived  fimm  standard  formulas  along  the 
Program  Manager’s  cstimatc-at-completion. 

Finding  C:  The  ADS  Program  Office  had  not  completed  actions  to  update  the  acquisition 
strategy;  cost  analysis  requirements  description;  life-cycle  cost  estimate;  C41  support  plan; 
and  programmatic  environmental,  safety,  and  health  evaluation  plan  as  required  to  show  the 
current  status  of  the  program.  This  condition  occurred  because  the  program  office  could  not 
update  program  documentation  because  an  updated  ORD  and  test  and  evaluation  master  plan 
were  not  yet  approved.  As  a  result,  the  program  manager  did  not  have  up-to-date  acquisition 
documentation  needed  to  effectively  maitage  program  cost  and  performance  and  acquisition 
decision  makers  could  not  make  fully  informed  investment  decisions. 

Recommendation  C;  We  recommend  that  the  program  manager  for  the  Advanced 
Deployable  System,  concurrent  with  the  approval  of  the  operational  requirements  document 
and  the  test  and  evaluation  master  plan: 

1 .  Update  the  acquisition  strategy  to  incorporate  the  restructured  four-block 
evolutionary  acquisition  strategy. 

2.  Update  the  cost  analysis  requirements  description  to  be  used  as  a  basis  for  the 
life-cycle  cost  estimate. 

3.  Update  the  life-cycle  cost  estimate,  including  disposal  costs,  for  all  four 
blocks  of  the  evolutionary  acquisition  strategy  and  direct  the  independent 
review  of  the  life-cycle  cost  estimate. 

4.  Update  the  commarKf,  control,  communications,  computers,  and  intelligence 

support  plan  to  document  required  interfaces  with  other  systems. 

5.  Update  the  programmatic,  environmental,  safety,  and  health  evaluation  plan  to 
show  that  the  system  can  be  tested,  operated,  maintained,  repaired,  and 
disposed  of  in  compliance  with  environmental  regulations. 

DoN  Response:  Cmicur.  Updating  of  the  acquisition  strategy  and  all  documentation  related 
of  the  ADS  program,  inclusive  of  all  four  blocks,  is  in  progress  and  will  be  completed  no 
later  than  28  February  2003.  The  program  office  will  also  complete  a  life  cycle  cost  estimate 
for  all  four  blocks  by  the  end  of  February  2003.  The  updates  to  the  C4I  support  plan  and 
environmental,  safety,  and  health  evaluation  plan  will  be  completed  no  later  than  the  31 
October  2002.  The  program  office  is  developing  a  plan  of  action  and  milestones  to  drive  and 
track  the  process  for  updating  all  needed  documentation  in  a  timely  fashion.  This  action  plan 
will  be  completed  prior  to  an  upcoming  program  review  with  the  Assistant  Secretary  of  the 
Navy  (Research,  Development  and  Acquisition)  currently  platmed  in  October  2002 


Enclosure  (1) 
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Team  Members 


The  Acquisition  Management  Directorate,  Office  of  the  Assistant  Inspector 
General  for  Auditing  of  the  Department  of  Defense  prepared  this  report. 
Personnel  of  the  Office  of  the  Inspector  General  of  the  Department  of  Defense 
who  contributed  to  the  report  are  listed  below. 

Mary  L.  Ugone 
John  E.  Meling 
Susan  J.  Lippolis 
Amy  L.  Mathews 
Michael  T.  Burger 
Lidet  K.  Negash 
Jacqueline  N.  Pugh 


